Introduction to the Special Issue "Applications in Self-Aware Computing Systems and their Evaluation"

The joint 1st Workshop on Evaluations and Measurements in Self-Aware Computing Systems (EMSAC 2019) and Workshop on Self-Aware Computing (SeAC) was held as part of the FAS* conference alliance in conjunction with the 16th IEEE International Conference on Autonomic Computing (ICAC) and the 13th IEEE International Conference on Self-Adaptive and Self-Organizing Systems (SASO) in Umeå, Sweden on 20 June 2019. The goal of this one-day workshop was to bring together researchers and practitioners from academic environments and from the industry to share their solutions, ideas, visions, and doubts in self-aware computing systems in general and in the evaluation and measurements of such systems in particular. The workshop aimed to enable discussions, partnerships, and collaborations among the participants. This special issue follows the theme of the workshop. It contains extended versions of workshop presentations as well as additional contributions

Model-Based Testing for Self-Organization Mechanisms

A complex system is a system composed of many different (often heterogeneous) com- ponents that interact with each other. These complex systems become more ubiquitous, e.g., in cloud computing, the internet of things, or industry 4.0. Self-Organization (SO) is offering new flexibility that can organize this complexity: Self-Organization enables a restructuring of a system and its components at run time in order to conform to the system’s dynamic and ever-changing environment and to fulfill its goals without human intervention. The complexity of the system arises either from the lack of knowledge at design time or the hardly predictable operational conditions of the systems at run time. However, the newly gained flexibility through the introduction of SO comes at a price: the design and implementation of SO mechanisms are a demanding engineering task. Whereas there is much research in the design and implementation of SO mechanisms, there is a lack of adequate testing techniques. This thesis narrows the gap by providing a Model-Based Testing (MBT) approach for SO mechanisms. Six significant contributions constitute the approach: Revealing failures—the intent of testing—demands a definition of the correct and the incorrect behavior of the investigated System under Test (SuT). In general, the correct behavior of an SuT is specified, and any deviation is not intended. Judging about failures is difficult in SO systems which might recover from a situation where the specification is only temporarily not fulfilled. This thesis thus offers a new notion of failure for SO mechanisms that copes with the specific behavior of SO mechanisms. Self-Organization mechanisms are highly interwoven with the controlled system and its environment. This characteristic makes it difficult to control and observe the during testing. With the architectural pattern of the Corridor Enforcing Infrastructure (CEI), this thesis offers a generic approach for making SO mechanisms testable. Further, we show that a broad class of SO systems is testable within this approach. Moreover, the concepts of the CEI allow for systematically decomposed and assembled SO mechanisms for isolated and integrated testing. The resulting test architecture, presented in this thesis, includes all these contributions and offers a complete test scaffold. Within this scaffold, an MBT approach for SO mechanisms is provided. This approach is suited for SO mechanisms by extending MBT with a so-called models at run time approach that allows for closing the feedback loop of MBT. Thus, it is possible to use the feedback of the Self-Organization Mechanism under Test (SOuT) during execution in the test model. This knowledge is exploited in a fault-based and a probabilistic model for test case generation. Within this model, we can apply a suitable search-based test generation approach for efficient testing of SO mechanisms. Thus, we can exploit unique characteristics of SO mechanisms to speed up failure detection by up to 500 times. Testing the functional requirements of an SO mechanisms is the focus of this thesis. However, we are also able to show the ability of the presented MBT approach to investigate the performance of SO mechanisms. In order to demonstrate the capabilities of the presented approach, it is evaluated on five different case studies incorporating different kinds of SO mechanisms that originate from different companies, institutions, and researchers

Model-Based Testing for Self-Organization Mechanisms

A complex system is a system composed of many different (often heterogeneous) com- ponents that interact with each other. These complex systems become more ubiquitous, e.g., in cloud computing, the internet of things, or industry 4.0. Self-Organization (SO) is offering new flexibility that can organize this complexity: Self-Organization enables a restructuring of a system and its components at run time in order to conform to the system’s dynamic and ever-changing environment and to fulfill its goals without human intervention. The complexity of the system arises either from the lack of knowledge at design time or the hardly predictable operational conditions of the systems at run time. However, the newly gained flexibility through the introduction of SO comes at a price: the design and implementation of SO mechanisms are a demanding engineering task. Whereas there is much research in the design and implementation of SO mechanisms, there is a lack of adequate testing techniques. This thesis narrows the gap by providing a Model-Based Testing (MBT) approach for SO mechanisms. Six significant contributions constitute the approach: Revealing failures—the intent of testing—demands a definition of the correct and the incorrect behavior of the investigated System under Test (SuT). In general, the correct behavior of an SuT is specified, and any deviation is not intended. Judging about failures is difficult in SO systems which might recover from a situation where the specification is only temporarily not fulfilled. This thesis thus offers a new notion of failure for SO mechanisms that copes with the specific behavior of SO mechanisms. Self-Organization mechanisms are highly interwoven with the controlled system and its environment. This characteristic makes it difficult to control and observe the during testing. With the architectural pattern of the Corridor Enforcing Infrastructure (CEI), this thesis offers a generic approach for making SO mechanisms testable. Further, we show that a broad class of SO systems is testable within this approach. Moreover, the concepts of the CEI allow for systematically decomposed and assembled SO mechanisms for isolated and integrated testing. The resulting test architecture, presented in this thesis, includes all these contributions and offers a complete test scaffold. Within this scaffold, an MBT approach for SO mechanisms is provided. This approach is suited for SO mechanisms by extending MBT with a so-called models at run time approach that allows for closing the feedback loop of MBT. Thus, it is possible to use the feedback of the Self-Organization Mechanism under Test (SOuT) during execution in the test model. This knowledge is exploited in a fault-based and a probabilistic model for test case generation. Within this model, we can apply a suitable search-based test generation approach for efficient testing of SO mechanisms. Thus, we can exploit unique characteristics of SO mechanisms to speed up failure detection by up to 500 times. Testing the functional requirements of an SO mechanisms is the focus of this thesis. However, we are also able to show the ability of the presented MBT approach to investigate the performance of SO mechanisms. In order to demonstrate the capabilities of the presented approach, it is evaluated on five different case studies incorporating different kinds of SO mechanisms that originate from different companies, institutions, and researchers

Model-Based Testing for Self-Organization Mechanisms

A complex system is a system composed of many different (often heterogeneous) com- ponents that interact with each other. These complex systems become more ubiquitous, e.g., in cloud computing, the internet of things, or industry 4.0. Self-Organization (SO) is offering new flexibility that can organize this complexity: Self-Organization enables a restructuring of a system and its components at run time in order to conform to the system’s dynamic and ever-changing environment and to fulfill its goals without human intervention. The complexity of the system arises either from the lack of knowledge at design time or the hardly predictable operational conditions of the systems at run time. However, the newly gained flexibility through the introduction of SO comes at a price: the design and implementation of SO mechanisms are a demanding engineering task. Whereas there is much research in the design and implementation of SO mechanisms, there is a lack of adequate testing techniques. This thesis narrows the gap by providing a Model-Based Testing (MBT) approach for SO mechanisms. Six significant contributions constitute the approach: Revealing failures—the intent of testing—demands a definition of the correct and the incorrect behavior of the investigated System under Test (SuT). In general, the correct behavior of an SuT is specified, and any deviation is not intended. Judging about failures is difficult in SO systems which might recover from a situation where the specification is only temporarily not fulfilled. This thesis thus offers a new notion of failure for SO mechanisms that copes with the specific behavior of SO mechanisms. Self-Organization mechanisms are highly interwoven with the controlled system and its environment. This characteristic makes it difficult to control and observe the during testing. With the architectural pattern of the Corridor Enforcing Infrastructure (CEI), this thesis offers a generic approach for making SO mechanisms testable. Further, we show that a broad class of SO systems is testable within this approach. Moreover, the concepts of the CEI allow for systematically decomposed and assembled SO mechanisms for isolated and integrated testing. The resulting test architecture, presented in this thesis, includes all these contributions and offers a complete test scaffold. Within this scaffold, an MBT approach for SO mechanisms is provided. This approach is suited for SO mechanisms by extending MBT with a so-called models at run time approach that allows for closing the feedback loop of MBT. Thus, it is possible to use the feedback of the Self-Organization Mechanism under Test (SOuT) during execution in the test model. This knowledge is exploited in a fault-based and a probabilistic model for test case generation. Within this model, we can apply a suitable search-based test generation approach for efficient testing of SO mechanisms. Thus, we can exploit unique characteristics of SO mechanisms to speed up failure detection by up to 500 times. Testing the functional requirements of an SO mechanisms is the focus of this thesis. However, we are also able to show the ability of the presented MBT approach to investigate the performance of SO mechanisms. In order to demonstrate the capabilities of the presented approach, it is evaluated on five different case studies incorporating different kinds of SO mechanisms that originate from different companies, institutions, and researchers